对象之间的良好距离和相似性度量的选择对于许多机器学习方法很重要。因此，近年来已经开发了许多度量学习算法，主要用于欧几里得数据，以提高分类或聚类方法的性能。但是，由于难以在归因图之间建立可计算，高效和可区分的距离，尽管社区的浓厚兴趣，但很少开发适合图形的度量学习算法。在本文中，我们通过提出一个新的简单图表学习 - SGML-模型，该模型几乎没有基于简单的图形卷积神经网络-SGCN-和最佳传输理论元素。该模型使我们能够与标记（属性）图的数据库建立适当的距离，以提高简单分类算法（例如$ k $ -nn）的性能。可以快速训练这个距离，同时保持良好的表现，如本文中提出的实验研究所示。

了解极端事件及其可能性是研究气候变化影响，风险评估，适应和保护生物的关键。在这项工作中，我们开发了一种方法来构建极端热浪的预测模型。这些模型基于卷积神经网络，对极长的8，000年气候模型输出进行了培训。由于极端事件之间的关系本质上是概率的，因此我们强调概率预测和验证。我们证明，深度神经网络适用于法国持续持续14天的热浪，快速动态驱动器提前15天（500 hpa地球电位高度场），并且在慢速较长的交货时间内，慢速物理时间驱动器（土壤水分）。该方法很容易实现和通用。我们发现，深神经网络选择了与北半球波数字3模式相关的极端热浪。我们发现，当将2米温度场添加到500 HPA地球电位高度和土壤水分场中时，2米温度场不包含任何新的有用统计信息。主要的科学信息是，训练深层神经网络预测极端热浪的发生是在严重缺乏数据的情况下发生的。我们建议大多数其他应用在大规模的大气和气候现象中都是如此。我们讨论了处理缺乏数据制度的观点，例如罕见的事件模拟，以及转移学习如何在后一种任务中发挥作用。

由于极端热波和热圆顶对社会和生物多样性的影响，他们的研究是一个关键挑战。我们专门研究了持久的极端热浪，这是气候影响最重要的热潮。物理驱动天气预报系统或气候模型可用于预测其发生或预测其概率。目前的工作探讨了使用深度学习架构的使用，使用气候模型的输出训练，作为预测极端持久热浪的发生的替代策略。这种新方法将对包括气候模型统计数据研究的几个关键科学目标，建立了对气候模型中罕见事件的定量代理，研究了气候变化的影响，并最终应对预测有用。履行这些重要目标意味着解决与罕见事件预测有本质相关的类大小不平衡的问题，评估转移学习的潜在好处，以解决极端事件的嵌套性质（自然包含在不太极端的情况下）。我们训练一个卷积神经网络，使用1000年的气候模型产出，具有大级欠采样和转移学习。从观察到的表面温度和500 HPA地球态高度场的快照，训练有素的网络在预测持久的极端热浪的发生时实现了显着性能。我们能够以三种不同的强度预测它们，早在活动开始前15天（事件结束前30天）。

我们介绍了一种新颖的谐波分析，用于在函数上定义的函数，随机步行操作员是基石。作为第一步，我们将随机步行操作员的一组特征向量作为非正交傅里叶类型的功能，用于通过定向图。我们通过将从其Dirichlet能量获得的随机步行操作员的特征向量的变化与其相关的特征值的真实部分连接来发现频率解释。从这个傅立叶基础，我们可以进一步继续，并在有向图中建立多尺度分析。通过将Coifman和MagGioni扩展到定向图，我们提出了一种冗余小波变换和抽取的小波变换。因此，我们对导向图的谐波分析的发展导致我们考虑应用于突出了我们框架效率的指示图的图形上的半监督学习问题和信号建模问题。

Recent advances in deep learning have enabled us to address the curse of dimensionality (COD) by solving problems in higher dimensions. A subset of such approaches of addressing the COD has led us to solving high-dimensional PDEs. This has resulted in opening doors to solving a variety of real-world problems ranging from mathematical finance to stochastic control for industrial applications. Although feasible, these deep learning methods are still constrained by training time and memory. Tackling these shortcomings, Tensor Neural Networks (TNN) demonstrate that they can provide significant parameter savings while attaining the same accuracy as compared to the classical Dense Neural Network (DNN). In addition, we also show how TNN can be trained faster than DNN for the same accuracy. Besides TNN, we also introduce Tensor Network Initializer (TNN Init), a weight initialization scheme that leads to faster convergence with smaller variance for an equivalent parameter count as compared to a DNN. We benchmark TNN and TNN Init by applying them to solve the parabolic PDE associated with the Heston model, which is widely used in financial pricing theory.

Training a very deep neural network is a challenging task, as the deeper a neural network is, the more non-linear it is. We compare the performances of various preconditioned Langevin algorithms with their non-Langevin counterparts for the training of neural networks of increasing depth. For shallow neural networks, Langevin algorithms do not lead to any improvement, however the deeper the network is and the greater are the gains provided by Langevin algorithms. Adding noise to the gradient descent allows to escape from local traps, which are more frequent for very deep neural networks. Following this heuristic we introduce a new Langevin algorithm called Layer Langevin, which consists in adding Langevin noise only to the weights associated to the deepest layers. We then prove the benefits of Langevin and Layer Langevin algorithms for the training of popular deep residual architectures for image classification.

Machine learning (ML) models can leak information about users, and differential privacy (DP) provides a rigorous way to bound that leakage under a given budget. This DP budget can be regarded as a new type of compute resource in workloads of multiple ML models training on user data. Once it is used, the DP budget is forever consumed. Therefore, it is crucial to allocate it most efficiently to train as many models as possible. This paper presents the scheduler for privacy that optimizes for efficiency. We formulate privacy scheduling as a new type of multidimensional knapsack problem, called privacy knapsack, which maximizes DP budget efficiency. We show that privacy knapsack is NP-hard, hence practical algorithms are necessarily approximate. We develop an approximation algorithm for privacy knapsack, DPK, and evaluate it on microbenchmarks and on a new, synthetic private-ML workload we developed from the Alibaba ML cluster trace. We show that DPK: (1) often approaches the efficiency-optimal schedule, (2) consistently schedules more tasks compared to a state-of-the-art privacy scheduling algorithm that focused on fairness (1.3-1.7x in Alibaba, 1.0-2.6x in microbenchmarks), but (3) sacrifices some level of fairness for efficiency. Therefore, using DPK, DP ML operators should be able to train more models on the same amount of user data while offering the same privacy guarantee to their users.

Imperfect information games (IIG) are games in which each player only partially observes the current game state. We study how to learn $\epsilon$-optimal strategies in a zero-sum IIG through self-play with trajectory feedback. We give a problem-independent lower bound $\mathcal{O}(H(A_{\mathcal{X}}+B_{\mathcal{Y}})/\epsilon^2)$ on the required number of realizations to learn these strategies with high probability, where $H$ is the length of the game, $A_{\mathcal{X}}$ and $B_{\mathcal{Y}}$ are the total number of actions for the two players. We also propose two Follow the Regularize leader (FTRL) algorithms for this setting: Balanced-FTRL which matches this lower bound, but requires the knowledge of the information set structure beforehand to define the regularization; and Adaptive-FTRL which needs $\mathcal{O}(H^2(A_{\mathcal{X}}+B_{\mathcal{Y}})/\epsilon^2)$ plays without this requirement by progressively adapting the regularization to the observations.

Stochastic Gradient Descent Langevin Dynamics (SGLD) algorithms, which add noise to the classic gradient descent, are known to improve the training of neural networks in some cases where the neural network is very deep. In this paper we study the possibilities of training acceleration for the numerical resolution of stochastic control problems through gradient descent, where the control is parametrized by a neural network. If the control is applied at many discretization times then solving the stochastic control problem reduces to minimizing the loss of a very deep neural network. We numerically show that Langevin algorithms improve the training on various stochastic control problems like hedging and resource management, and for different choices of gradient descent methods.

Neural machine translation (NMT) has become the de-facto standard in real-world machine translation applications. However, NMT models can unpredictably produce severely pathological translations, known as hallucinations, that seriously undermine user trust. It becomes thus crucial to implement effective preventive strategies to guarantee their proper functioning. In this paper, we address the problem of hallucination detection in NMT by following a simple intuition: as hallucinations are detached from the source content, they exhibit encoder-decoder attention patterns that are statistically different from those of good quality translations. We frame this problem with an optimal transport formulation and propose a fully unsupervised, plug-in detector that can be used with any attention-based NMT model. Experimental results show that our detector not only outperforms all previous model-based detectors, but is also competitive with detectors that employ large models trained on millions of samples.